Analysis of marketing and entertainment effectiveness using magnetoencephalography

ABSTRACT

Central nervous system, autonomic nervous system, and effector data is measured and analyzed to determine the effectiveness of marketing and entertainment stimuli. A data collection mechanism including multiple modalities such as Magnetoencephalography (MEG), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data. The response data is enhanced using intra-modality response synthesis and/or a cross-modality response synthesis.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to Provisional Patent Application60/973,917 (Docket No. 2007NF2) titled Advertising, Marketing,Entertainment And Brand Effectiveness Analyzer Utilizing MEG IncludedCentral Nervous System, Autonomous Nervous System And Effector SystemMeasurements by Anantha Pradeep, Robert T. Knight, and RamachandranGurumoorthy, and filed on Sep. 20, 2007.

TECHNICAL FIELD

The present disclosure relates to the analysis of the effectiveness ofmarketing and entertainment using Magnetoencephalography (MEG) and othercentral nervous system, autonomic nervous system, and effectormeasurement mechanisms.

DESCRIPTION OF RELATED ART

Conventional systems for measuring the effectiveness of entertainmentand marketing including advertising, brand messages, and productplacement rely on either survey based evaluations or limitedneurophysiological measurements used in isolation. These conventionalsystems provide some useful data but are highly inefficient andinaccurate due to a variety of semantic, syntactic, metaphorical,cultural, social, and interpretative errors and biases. The systems andtechniques themselves used to obtain neurophysiological measurements arealso highly limited.

Consequently, it is desirable to provide improved methods and apparatusfor measuring and analyzing neurological and neurophysiological data,such as central nervous system, autonomic nervous system, and effectordata obtained during evaluation of the effectiveness of entertainmentand marketing materials.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings, whichillustrate particular example embodiments.

FIG. 1 illustrates one example of a system for determining theeffectiveness of marketing and entertainment by using central nervoussystem, autonomic nervous system, and effector measures.

FIG. 2 illustrates a particular example of a system having anintelligent protocol generator and presenter device and individualmechanisms for intra-modality response synthesis.

FIG. 3 illustrates a particular example of an intra-modality synthesismechanism for Magnetoencephalography (MEG).

FIG. 4 illustrates another particular example of synthesis forMagnetoencephalography (MEG).

FIG. 5 illustrates a particular example of a cross-modality synthesismechanism.

FIG. 6 is one example of a sample flow process diagram showing atechnique for obtaining neurological and neurophysiological data.

FIG. 7 provides one example of a system that can be used to implementone or more mechanisms.

DESCRIPTION OF PARTICULAR EMBODIMENTS

Reference will now be made in detail to some specific examples of theinvention including the best modes contemplated by the inventors forcarrying out the invention. Examples of these specific embodiments areillustrated in the accompanying drawings. While the invention isdescribed in conjunction with these specific embodiments, it will beunderstood that it is not intended to limit the invention to thedescribed embodiments. On the contrary, it is intended to coveralternatives, modifications, and equivalents as may be included withinthe spirit and scope of the invention as defined by the appended claims.

For example, the techniques and mechanisms of the present invention willbe described in the context of evaluating entertainment and marketingeffectiveness. However, it should be noted that the techniques andmechanisms of the present invention apply to a variety of differenttypes of entertainment and marketing such as video and audio streams,media advertising, product placement, brand effectiveness, printedadvertisements, etc. It should be noted that various mechanisms andtechniques can be applied to any type of stimuli. In the followingdescription, numerous specific details are set forth in order to providea thorough understanding of the present invention. Particular exampleembodiments of the present invention may be implemented without some orall of these specific details. In other instances, well known processoperations have not been described in detail in order not tounnecessarily obscure the present invention.

Various techniques and mechanisms of the present invention willsometimes be described in singular form for clarity. However, it shouldbe noted that some embodiments include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. For example, a system uses a processor in a variety ofcontexts. However, it will be appreciated that a system can use multipleprocessors while remaining within the scope of the present inventionunless otherwise noted. Furthermore, the techniques and mechanisms ofthe present invention will sometimes describe a connection between twoentities. It should be noted that a connection between two entities doesnot necessarily mean a direct, unimpeded connection, as a variety ofother entities may reside between the two entities. For example, aprocessor may be connected to memory, but it will be appreciated that avariety of bridges and controllers may reside between the processor andmemory. Consequently, a connection does not necessarily mean a direct,unimpeded connection unless otherwise noted.

Overview

Central nervous system, autonomic nervous system, and effector data ismeasured and analyzed to determine the effectiveness of marketing andentertainment stimuli. A data collection mechanism including multiplemodalities such as Magnetoencephalography (MEG), Electrooculography(EOG), Galvanic Skin Response (GSR), etc., collects response data fromsubjects exposed to marketing and entertainment stimuli. A data cleansermechanism filters the response data. The response data is enhanced usingintra-modality response synthesis and/or a cross-modality responsesynthesis.

Example Embodiments

Conventional mechanisms for obtaining information about theeffectiveness of various types of stimuli such as marketing andentertainment materials have generally relied on focus groups andsurveys. Subjects are provided with oral and written mechanisms forconveying their thoughts and feelings elicited in response to aparticular advertisement, brand, media clip, etc. These oral and writtenmechanisms provide some limited information on the effectiveness of themarketing and entertainment materials, but have a variety oflimitations. For example, subjects may be unable or unwilling to expresstheir true thoughts and feelings about a topic, or questions may bephrased with built in bias. Articulate subjects may be given more weightthan nonexpressive ones. A variety of semantic, syntactic, metaphorical,cultural, social and interpretive biases and errors prevent accurate andrepeatable evaluation.

Some efforts have been made to use isolated neurological andneurophysiological measurements to gauge subject responses. Someexamples of central nervous system measurement mechanisms includeFunctional Magnetic Resonance Imaging (fMRI) and Magnetoencephalography(MEG). fMRI measures blood oxygenation in the brain that correlates withincreased neural activity. However, current implementations of fMRI havepoor temporal resolution of few seconds. MEG measures electricalactivity associated with post synaptic currents occurring in themilliseconds range. MEG provides an electromagnetic measurement ofneural activity generated by coherent ensembles of neurons. Similar toElectroencephalography (EEG), MEG provides precise temporal measures ofneural activity. MEG, however, has further benefits such as providingenhanced dipole localization relative to EEG. Thus MEG provides anelectromagnetic measurement technique that provides both temporally andspatially localized measures of neural activity.

In addition to localizing regional activations, the MEG activity can bedivided into time-frequency analyses of on-going MEG and extraction ofstimulus or response locked Event-Related-Potential or Event RelatedPower Spectrum Perturbations.Subcranial MEG can measure electricalactivity with the most accuracy, as the bone and dermal layers weakentransmission of a wide range of frequencies. Nonetheless, surface MEGprovides a wealth of electrophysiological information if analyzedproperly.

Autonomic nervous system measurement mechanisms include Galvanic SkinResponse (GSR), Electrocardiograms (EKG), pupillary dilation, etc.Effector measurement mechanisms include Electrooculography (EOG), eyetracking, facial emotion encoding, reaction time, etc.

Some conventional mechanisms cite a particular neurological orneurophysiological measurement characteristic as indicating a particularthought, feeling, mental state, or ability. For example, one mechanismpurports that the contraction of a particular facial muscle indicatesthe presence of a particular emotion. Others measure general activity inparticular areas of the brain and suggest that activity in one portionmay suggest lying while activity in another portion may suggesttruthfulness. However, these mechanisms are severely limited in theirability to accurately reflect a subject's actual thoughts. It isrecognized that a particular region of the brain can not be mapped to aparticular thought. Similarly, a particular eye movement can not bemapped to a particular emotion. Even when there is a strong correlationbetween a particular measured characteristic and a thought, feeling, ormental state, the correlations are not perfect, leading to a largenumber of false positives and false negatives.

Consequently, the techniques and mechanisms of the present inventionintelligently blend multiple modes and manifestations of precognitiveneural signatures with cognitive neural signatures and post cognitiveneurophysiological manifestations to more accurately access theeffectiveness of marketing and entertainment materials. In someexamples, autonomic nervous system measures are themselves used tovalidate central nervous system measures. Effector and behaviorresponses are blended and combined with other measures.

Intra-modality measurement enhancements are made in addition to thecross-modality measurement mechanism enhancements. According to variousembodiments, brain activity is measured not just to determine theregions of activity, but to determine interactions and types ofinteractions between various regions. The techniques and mechanisms ofthe present invention recognize that interactions between neural regionssupport orchestrated and organized behavior. Thoughts and abilities arenot merely based on one part of the brain but instead rely on networkinteractions between brain regions.

The techniques and mechanisms of the present invention further recognizethat different frequency bands used for multi-regional communication canbe indicative of the effectiveness of stimuli. For example, associatinga name to a particular face may entail activity in communicationpathways tuned to particular frequencies. According to variousembodiments, select frequency bands are analyzed after filtering. Thetechniques and mechanisms of the present invention also recognize thathigh gamma band frequencies have significance. Inter-frequency couplingin the signals have also been determined to indicate effectiveness.Signals modulated on a carrier wave have also been determined to beimportant in evaluating thoughts and actions. In particular embodiments,the types of frequencies measured are subject and/or task specific. Forexample, particular types of frequencies in specific pathways aremeasured if a subject is being exposed to a new product.

In particular embodiments, evaluations are calibrated to each subjectand synchronized across subjects. In particular embodiments, templatesare created for subjects to create a baseline for measuring pre and poststimulus differentials. According to various embodiments, stimulusgenerators are intelligent, and adaptively modify specific parameterssuch as exposure length and duration for each subject being analyzed.

Consequently, the techniques and mechanisms of the present inventionprovide a central nervous system, autonomic nervous system, and effectormeasurement and analysis system that can be applied to evaluate theeffectiveness of materials such as marketing and entertainmentmaterials. Marketing materials may include advertisements, commercials,media clips, brand messages, product brochures, company logos, etc. Anintelligent stimulus generation mechanism intelligently adapts outputfor particular users and purposes. A variety of modalities can be usedincluding MEG, GSR, EKG, pupillary dilation, EOG, eye tracking, facialemotion encoding, reaction time, etc. Individual modalities such as MEGare enhanced by intelligently recognizing neural region communicationpathways. Cross modality analysis is enhanced using a synthesis andanalytical blending of central nervous system, autonomic nervous system,and effector signatures. Synthesis and analysis by mechanisms such astime and phase shifting, correlating, and validating intra-modaldeterminations allow generation of a composite output characterizing theeffectiveness of various stimuli.

FIG. 1 illustrates one example of a system for determining theeffectiveness of marketing and entertainment by using central nervoussystem, autonomic nervous system, and effector measures. According tovarious embodiments, the neuroanalysis system includes a protocolgenerator and presenter device 101. In particular embodiments, theprotocol generator and presenter device 101 is merely a presenter deviceand merely presents stimuli to a user. The stimuli may be a media clip,a commercial, a brand image, a magazine advertisement, a movie, an audiopresentation, particular tastes, smells, textures and/or sounds. Thestimuli can involve a variety of senses and occur with or without humansupervision. Continuous and discrete modes are supported. According tovarious embodiments, the protocol generator and presenter device 101also has protocol generation capability to allow intelligentcustomization of stimuli provided to a subject.

According to various embodiments, the subjects 103 are connected to datacollection devices 105. The data collection devices 105 may include avariety of neurological and neurophysiological measurement mechanismssuch as MEG, EOG, GSR, EKG, pupillary dilation, eye tracking, facialemotion encoding, and reaction time devices, etc. In particularembodiments, the data collection devices 105 include MEG 111, EOG 113,and GSR 115. In some instances, only a single data collection device isused. Data collection may proceed with or without human supervision.

The data collection device 105 collects neuro-physiological data frommultiple sources. This includes a combination of devices such as centralnervous system sources (MEG), autonomic nervous system sources (GSR,EKG, pupillary dilation), and effector sources (EOG, eye tracking,facial emotion encoding, reaction time). In particular embodiments, datacollected is digitally sampled and stored for later analysis. Inparticular embodiments, the data collected could be analyzed inreal-time. According to particular embodiments, the digital samplingrates are adaptively chosen based on the neurophysiological andneurological data being measured.

In one particular embodiment, the neurological and neurophysiologicalanalysis system includes MEG 111 measurements made using scalp levelelectrodes, EOG 113 measurements made using shielded electrodes to trackeye data, GSR 115 measurements performed using a differentialmeasurement system, a facial muscular measurement through shieldedelectrodes placed at specific locations on the face, and a facial affectgraphic and video analyzer adaptively derived for each individual.

In particular embodiments, the data collection devices are clocksynchronized with a protocol generator and presenter device 101. Thedata collection system 105 can collect data from a single individual (1system), or can be modified to collect synchronized data from multipleindividuals (N+1 system). The N+1 system may include multipleindividuals synchronously tested in isolation or in a group setting. Inparticular embodiments, the data collection devices also include acondition evaluation subsystem that provides auto triggers, alerts andstatus monitoring and visualization components that continuously monitorthe status of the subject, data being collected, and the data collectioninstruments. The condition evaluation subsystem may also present visualalerts and automatically trigger remedial actions.

According to various embodiments, the neurological andneurophysiological analysis system also includes a data cleanser device121. In particular embodiments, the data cleanser device 121 filters thecollected data to remove noise, artifacts, and other irrelevant datausing fixed and adaptive filtering, weighted averaging, advancedcomponent extraction (like PCA, ICA), vector and component separationmethods, etc. This device cleanses the data by removing both exogenousnoise (where the source is outside the physiology of the subject) andendogenous artifacts (where the source could be neurophysiological likemuscle movement, eye blinks, etc.).

The artifact removal subsystem includes mechanisms to selectivelyisolate and review the response data and identify epochs with timedomain and/or frequency domain attributes that correspond to artifactssuch as line frequency, eye blinks, and muscle movements. The artifactremoval subsystem then cleanses the artifacts by either omitting theseepochs, or by replacing these epoch data with an estimate based on theother clean data (for example, an MEG nearest neighbor weightedaveraging approach).

According to various embodiments, the data cleanser device 121 isimplemented using hardware, firmware, and/or software. It should benoted that although a data cleanser device 121 is shown located after adata collection device 105 and before synthesis devices 131 and 141, thedata cleanser device 121 like other components may have a location andfunctionality that varies based on system implementation. For example,some systems may not use any automated data cleanser device whatsoever.In other systems, data cleanser devices may be integrated intoindividual data collection devices.

The data cleanser device 121 passes data to the intra-modality responsesynthesizer 131. The intra-modality response synthesizer 131 isconfigured to customize and extract the independent neurological andneurophysiological parameters for each individual in each modality andblend the estimates within a modality analytically to elicit an enhancedresponse to the presented stimuli. In particular embodiments, theintra-modality response synthesizer also aggregates data from differentsubjects in a dataset.

According to various embodiments, the cross-modality response synthesisor fusion device 141 blends different intra-modality responses,including raw signals and signals output from synthesizer 131. Thecombination of signals enhances the measures of effectiveness within amodality. The cross-modality response fusion device 141 can alsoaggregate data from different subjects in a dataset.

According to various embodiments, the system also includes a compositeenhanced effectiveness estimator (CEEE) 153 that combines the enhancedresponses and estimates from each modality to provide a blended estimateof the effectiveness of the marketing and entertainment stimuli forvarious purposes. Stimulus effectiveness measures are output at 161.

FIG. 2 illustrates a particular example of a system having anintelligent protocol generator and presenter device (where theintelligence could include a feedback based on prior responses) andindividual mechanisms for intra-modality response synthesis.

According to various embodiments, the system includes a protocolgenerator and presenter device 201. In particular embodiments, theprotocol generator and presenter device 201 is merely a presenter deviceand merely presents preconfigured stimuli to a user. The stimuli may bemedia clips, commercials, brand images, magazine advertisements, movies,audio presentations, particular tastes, textures, smells, and/or sounds.The stimuli can involve a variety of senses and occur with or withouthuman supervision. Continuous and discrete modes are supported.According to various embodiments, the protocol generator and presenterdevice 201 also has protocol generation capability to allow intelligentmodification of the types of stimuli provided to a subject. Inparticular embodiments, the protocol generator and presenter device 201receives information about stimulus effectiveness measures fromcomponent 261.

The protocol generator and presenter device 201 dynamical adapts stimulipresentation by using information from the analysis of attention,analysis of emotional engagement, analysis of memory retention, analysisof overall visual, audio, other sensory effectiveness, and ad, show, orcontent effectiveness, implicit analysis of brand impact, implicitanalysis of brand meaning, implicit analysis of brand archetype,implicit analysis of brand imagery, implicit analysis of brand words,explicit analysis of brand impact, explicit analysis of brand meaning,explicit analysis of brand archetype, explicit analysis of brandimagery, explicit analysis of brand words; analysis of characters in thead, analysis of emotive response to characters in the ad/show/content,analysis of character interaction in the ad/show/content; elicitation ofcore components of the ad/show/content for print purposes, elicitationof core components of the ad/show/content for billboard purposes;elicitation of the ocular metrics like hot-zones in the ad/show/contentby eye dwell time, micro and macro saccade separation, saccadic returnsto points of interest; elicitation of points for product placement,elicitation of points for logo and brand placement; analysis of gameeffectiveness, analysis of product placement in games; analysis ofwebsite effectiveness, webpage dropoff in a site. According to variousembodiments, the information is provided by component 261. In particularembodiments, the protocol generator and presenter device 201 can itselfobtain some of this information

The protocol generator and presenter device 201 uses a data model alongwith linguistic and image tools like valence, arousal, meaning matchedword/phrase generators, valence and arousal matched image/videoselectors to generate parameters regarding the experiment. In particularexamples, the protocol generator and presenter device 201 may varyindividual presentation parameters like time and duration of theexperiment, the number of repetitions of the stimuli based on signal tonoise requirements, and the number and repetitions of the stimuli forhabituation and wear-out studies, the type and number ofneuro-physiological baselines, and the self reporting surveys toinclude.

In particular examples, the protocol generator and presenter device 201customizes presentations to a group of subjects or to individualsubjects. According to various embodiments, the subjects are connectedto data collection devices 205. The data collection devices 205 mayinvolve any type of neurological and neurophysiological mechanism suchas MEG, EOG, GSR, EKG, pupillary dilation, eye tracking, facial emotionencoding, reaction rime, etc. In particular embodiments, the datacollection devices 205 include MEG 211, EOG 213, and GSR 215. In someinstances, only a single modality is used. In other instances, multiplemodalities are used and may vary depending on the type of effectivenessevaluation. Data collection may proceed without or without humansupervision.

The data collection device 205 automatically collectsneuro-physiological data from multiple sources. This includes acombination of devices such as central nervous system sources (MEG),autonomic nervous system sources (GSR, EKG, pupillary dilation), andeffector sources (EOG, eye tracking, facial emotion encoding, reactiontime). In particular embodiments, data collected is digitally sampledand stored for later analysis. The digital sampling rates are adaptivelychosen based on the type of neurophysiological and neurological databeing measured.

In particular embodiments, the system includes MEG 211 measurements madeusing scalp level electrodes, EOG 213 measurements made using shieldedelectrodes to track eye data, GSR 215 measurements performed using adifferential measurement system, and a facial affect graphic and videoanalyzer adaptively derived for each individual.

According to various embodiments, the data collection devices are clocksynchronized with a protocol generator and presenter device 201. Thedata collection system 205 can collect data from a single individual (1system), or can be modified to collect synchronized data from multipleindividuals (N+1 system). The N+1 system could include multipleindividuals synchronously recorded in a group setting or in isolation.In particular embodiments, the data collection devices also include acondition evaluation subsystem that provides auto triggers, alerts andstatus monitoring and visualization components that continuously monitorthe status of the data being collected as well as the status of the datacollection instruments themselves. The condition evaluation subsystemmay also present visual alerts and automatically trigger remedialactions.

According to various embodiments, the system also includes a datacleanser device 221. In particular embodiments, the data cleanser device221 filters the collected data to remove noise, artifacts, and otherirrelevant data using fixed and adaptive filtering, weighted averaging,advanced component extraction (like PCA, ICA), vector and componentseparation methods, etc. This device cleanses the data by removing bothexogenous noise (where the source is outside the physiology of thesubject) and endogenous artifacts (where the source could beneurophysiological like muscle movement, eye blinks).

The artifact removal subsystem includes mechanisms to selectivelyisolate and review the output of each of the data and identify epochswith time domain and/or frequency domain attributes that correspond toartifacts such as line frequency, eye blinks, and muscle movements. Theartifact removal subsystem then cleanses the artifacts by eitheromitting these epochs, or by replacing these epoch data with an estimatebased on the other clean data (for example, an MEG nearest neighborweighted averaging approach), or removes these components from thesignal.

According to various embodiments, the data cleanser device 221 isimplemented using hardware, firmware, and/or software. It should benoted that although a data cleanser device 221 is shown located after adata collection device 205 and before synthesis devices 231 and 241, thedata cleanser device 221 like other components may have a location andfunctionality that varies based on system implementation. For example,some systems may not use any automated data cleanser device whatsoever.In other systems, data cleanser devices may be integrated intoindividual data collection devices.

The data cleanser device 221 passes data to the intra-modality responsesynthesizer 231. The intra-modality response synthesizer is configuredto customize and extract the independent neurological andneurophysiological parameters for each individual in each modality andblend the estimates within a modality analytically to elicit an enhancedresponse to the presented stimuli. In particular embodiments, theintra-modality response synthesizer also aggregates data from differentsubjects in a dataset. According to various embodiments, various modulesperform synthesis in parallel or in series, and can operate on datadirectly output from a data cleanser device 221 or operate on dataoutput from other modules. For example, MEG synthesis module 233 canoperate on the output of EOG synthesis module 235. GSR module 237 canoperate on data output from MEG module 233.

According to various embodiments, the cross-modality response synthesisor fusion device 241 blends different intra-modality responses,including raw signals as well as signals output from synthesizer 231.The combination of signals enhances the measures of effectiveness withina modality. The cross-modality response fusion device 241 can alsoaggregate data from different subjects in a dataset.

According to various embodiments, the neuro analysis system alsoincludes a composite enhanced effectiveness estimator (CEEE) 251 thatcombines the enhanced responses and estimates from each modality toprovide a blended estimate of the effectiveness of the marketing andadvertising stimuli for various purposes. Stimulus effectivenessmeasures are output at 261. A portion or all of the effectivenessmeasures (intra-modality synthesizer, cross modality fusion device,and/or the CEEE) can be provided as feedback to a protocol generator andpresenter device 201 to further customize stimuli presented to users203.

FIG. 3 illustrates a particular example of an intra-modality synthesismechanism. In particular embodiments, MEG response data is synthesizedto provide an enhanced assessment of marketing and entertainmenteffectiveness. According to various embodiments, MEG measures electricalactivity resulting from thousands of simultaneous neural processesassociated with different portions of the brain. MEG data can beclassified in various bands. According to various embodiments, brainwavefrequencies includes delta, theta, alpha, beta, and gamma frequencyranges. Delta waves are classified as those less than 4 Hz and areprominent during deep sleep. Theta waves have frequencies between 3.5 to7.5 Hz and are associated with memories, attention, emotions, andsensations. Theta waves are typically prominent during states ofinternal focus.

Alpha frequencies reside between 7.5 and 13 Hz and typically peak around10 Hz. Alpha waves are prominent during states of relaxation. Beta waveshave a frequency range between 14 and 30 Hz. Beta waves are prominentduring states of motor control, long range synchronization between brainareas, analytical problem solving, judgment, and decision making. Gammawaves occur between 30 and 60 Hz and are involved in binding ofdifferent populations of neurons together into a network for the purposeof carrying out a certain cognitive or motor function, as well as inattention and memory. Because the skull and dermal layers attenuatewaves in this frequency range, brain waves above 75-80 Hz are difficultto detect and are often not used for stimuli response assessment.

However, the techniques and mechanisms of the present inventionrecognize that analyzing high gamma band (kappa-band: Above 60 Hz)measurements, in addition to theta, alpha, beta, and low gamma bandmeasurements, enhances neurological attention, emotional engagement andretention component estimates. In particular embodiments, MEGmeasurements including difficult to detect high gamma or kappa bandmeasurements are obtained, enhanced, and evaluated at 301. At 303,subject and task specific signature sub-bands in the theta, alpha, beta,gamma and kappa bands are identified to provide enhanced responseestimates. According to various embodiments, high gamma waves(kappa-band) above 80 Hz (typically detectable with sub-cranial MEG andmagnetoencephalograophy) can be used in inverse model-based enhancementof the frequency responses to the stimuli.

Various embodiments of the present invention recognize that particularsub-bands within each frequency range have particular prominence duringcertain activities. A subset of the frequencies in a particular band isreferred to herein as a sub-band. For example, a sub-band may includethe 40-45 Hz range within the gamma band. In particular embodiments,multiple sub-bands within the different bands are selected whileremaining frequencies are band pass filtered. In particular embodiments,multiple sub-band responses may be enhanced, while the remainingfrequency responses may be attenuated.

At 305, inter-regional coherencies of the sub-band measurements aredetermined. According to various embodiments, inter-regional coherenciesare determined using gain and phase coherences, Bayesian references,mutual information theoretic measures of independence anddirectionality, and Granger causality techniques of the MEG response inthe different bands. In particular embodiments, inter-regionalcoherencies are determined using fuzzy logic to estimate effectivenessof the stimulus in evoking specific type of responses in individualsubjects.

At 307, inter-hemispheric time-frequency measurements are evaluated. Inparticular embodiments, asymmetries in specific band powers, asymmetriesin inter-regional intra-hemispheric coherences, and asymmetries ininter-regional intra-hemisphere inter-frequency coupling are analyzed toprovide measures of emotional engagement.

At 309, inter-frequency coupling assessments of the response aredetermined. In particular embodiments, a coupling index corresponding tothe measure of specific band activity in synchrony with the phase ofother band activity is determined to ascertain the significance of themarketing and advertising stimulus or sub-sections thereof. At 313, areference scalp power frequency curve is determined using a baselineelectrocorticogram (ECoG) power by frequency function driven model. Thereference scale power frequency curve is compared to an individual scalprecord power by frequency curve to derive scaled estimates of marketingand entertainment effectiveness. According to various embodiments,scaled estimates are derived used fuzzy scaling.

At 315, an information theory based band-weighting model is used foradaptive extraction of selective dataset specific, subject specific,task specific bands to enhance the effectiveness measure. Adaptiveextraction may be performed using fuzzy scaling. At 321, stimuli can bepresented and enhanced measurements determined multiple times todetermine the variation or habituation profiles across multiplepresentations. Determining the variation and/or habituation profilesprovides an enhanced assessment of the primary responses as well as thelongevity (wear-out) of the marketing and entertainment stimuli. At 323,the synchronous response of multiple individuals to stimuli presented inconcert is measured to determine an enhanced across subject synchronymeasure of effectiveness. According to various embodiments, thesynchronous response may be determined for multiple subjects residing inseparate locations or for multiple subjects residing in the samelocation.

Although a variety of synthesis mechanisms are described, it should berecognized that any number of mechanisms can be applied in sequence orin parallel with or without interaction between the mechanisms. In someexamples, processes 321 and 323 can be applied to any modality. FIG. 4illustrates a particular example of synthesis for Magnetoencephalography(MEG) data, including ERP and continuous MEG.

ERPs can be reliably measured using magnetoencephalography (MEG), aprocedure that measures electrical activity of the brain. Although anMEG reflects thousands of simultaneously ongoing brain processes, thebrain response to a certain stimulus may not be visible using MEG. ERPdata includes cognitive neurophysiological responses that manifestsafter the stimulus is presented. In many instances, it is difficult tosee an ERP after the presentation of a single stimulus. The most robustERPs are seen after tens or hundreds of individual presentations arecombined. This combination removes noise in the data and allows thevoltage response to the stimulus to stand out more clearly. In additionto averaging the embodiment includes techniques to extract single trialevoked information from the ongoing MEG.

While evoked potentials reflect the processing of the physical stimulus,event-related potentials are caused by the “higher” processes, thatmight involve memory, expectation, attention, or changes in the mentalstate, among others. According to various embodiments, evidence of theoccurrence or non-occurrence of specific time domain components inspecific regions of the brain are used to measure subject responsivenessto specific stimulus.

According to various embodiments, ERP data can be enhanced using avariety of mechanisms. At 401, event related time-frequency analysis ofstimulus response—event related power spectral perturbations (ERPSPs)—isperformed across multiple frequency bands such as theta, delta, alpha,beta, gamma and high gamma (kappa). According to various embodiments, abaseline ERP is determined. At 403, a differential event relatedpotential (DERP) is evaluated to assess stimulus attributabledifferential responses.

At 405, a variety of analysis techniques including principal componentanalysis (PCA), independent component analysis (ICA), and Monte Carlosanalysis can be applied to evaluate an ordered ranking of theeffectiveness across multiple stimuli. In particular embodiments, PCA isused to reduce multidimensional data sets to lower dimensions foranalysis. ICA is typically used to separate multiple components in asignal. Monte Carlo relies on repeated random sampling to computeresults. According to various embodiments, an ERP scenario is developedat 407 to determine a subject, session and task specific responsebaseline. The baseline can then be used to enhance the sensitivity ofother ERP responses to the tested stimuli.

At 421, stimuli can be presented and enhanced measurements determinedmultiple times to determine the variation or habituation profiles acrossmultiple presentations. Determining the variation and/or habituationprofiles provides an enhanced assessment of the primary responses aswell as the longevity (wear-out) of the marketing and entertainmentstimuli. At 423, the synchronous response of multiple individuals tostimuli presented in concert is measured to determine an enhanced acrosssubject synchrony measure of effectiveness. According to variousembodiments, the synchronous response may be determined for multiplesubjects residing in separate locations or for multiple subjectsresiding in the same location.

A variety of processes such as processes 421, and 423 can be applied toa number of modalities, including EOG, eye tracking, GSR, facial emotionencoding, etc. In addition, synthesis of data from mechanisms such asEOG and eye tracking can also benefit from the grouping objects ofinterest into temporally and spatially defined entities using micro andmacro saccade patterns. Gaze, dwell, return of eye movements toprimarily center around the defined entities of interest and inhibitionof return to novel regions of the material being evaluated are measuredto determine the degree of engagement and attention evoked by thestimulus.

Although intra-modality synthesis mechanisms provide enhancedeffectiveness data, additional cross-modality synthesis mechanisms canalso be applied. FIG. 5 illustrates a particular example of across-modality synthesis mechanism 521. A variety of mechanisms such asMEG 501, Eye Tracking 503, GSR 505, EOG 507, and facial emotion encoding509 are connected to a cross-modality synthesis mechanism. Othermechanisms as well as variations and enhancements on existing mechanismsmay also be included. According to various embodiments, data from aspecific modality can be enhanced using data from one or more othermodalities. In particular embodiments, MEG typically makes frequencymeasurements in different bands like alpha, beta and gamma to provideestimates of effectiveness. However, the techniques of the presentinvention recognize that effectiveness measures can be enhanced furtherusing information from other modalities.

For example, facial emotion encoding measures can be used to enhance thevalence of the MEG emotional engagement measure. EOG and eye trackingsaccadic measures of object entities can be used to enhance the MEGestimates of effectiveness including but not limited to attention,emotional engagement, and memory retention. According to variousembodiments, a cross-modality synthesis mechanism performs time andphase shifting of data to allow data from different modalities to align.In some examples, it is recognized that an MEG response will often occurhundreds of milliseconds before a facial emotion measurement changes.Correlations can be drawn and time and phase shifts made on anindividual as well as a group basis. In other examples, saccadic eyemovements may be determined as occurring before and after particular MEGresponses. According to various embodiments, time corrected GSR measuresare used to scale and enhance the MEG estimates of effectivenessincluding attention, emotional engagement and memory retention measures.

Evidence of the occurrence or non-occurrence of specific time domaindifference event-related potential components (like the DERP) inspecific regions correlates with subject responsiveness to specificstimulus. According to various embodiments, ERP measures are enhancedusing MEG time-frequency measures (ERPSP) in response to thepresentation of the marketing and entertainment stimuli. Specificportions are extracted and isolated to identify ERP, DERP and ERPSPanalyses to perform. In particular embodiments, an MEG frequencyestimation of attention, emotion and memory retention (ERPSP) is used asa co-factor in enhancing the ERP, DERP and time-domain responseanalysis.

EOG measures saccades to determine the presence of attention to specificobjects of stimulus. Eye tracking measures the subject's gaze path,location and dwell on specific objects of stimulus. According to variousembodiments, EOG and eye tracking is enhanced by measuring the presenceof lambda waves (a neurophysiological index of saccade effectiveness) inthe ongoing MEG in the occipital and extra striate regions, triggered bythe slope of saccade-onset to estimate the effectiveness of the EOG andeye tracking measures. In particular embodiments, specific MEGsignatures of activity such as slow potential shifts and measures ofcoherence in time-frequency responses at the Frontal Eye Field (FEF)regions that preceded saccade-onset are measured to enhance theeffectiveness of the saccadic activity data.

GSR typically measures the change in general arousal in response tostimulus presented. According to various embodiments, GSR is enhanced bycorrelating MEG/ERP responses and the GSR measurement to get an enhancedestimate of subject engagement. The GSR latency baselines are used inconstructing a time-corrected GSR response to the stimulus. Thetime-corrected GSR response is co-factored with the MEG measures toenhance GSR effectiveness measures.

According to various embodiments, facial emotion encoding uses templatesgenerated by measuring facial muscle positions and movements ofindividuals expressing various emotions prior to the testing session.These individual specific facial emotion encoding templates are matchedwith the individual responses to identify subject emotional response. Inparticular embodiments, these facial emotion encoding measurements areenhanced by evaluating inter-hemispherical asymmetries in MEG responsesin specific frequency bands and measuring frequency band interactions.The techniques of the present invention recognize that not only areparticular frequency bands significant in MEG responses, but particularfrequency bands used for communication between particular areas of thebrain are significant. Consequently, these MEG responses enhance theEMG, graphic and video based facial emotion identification.

FIG. 6 is a flow process diagram showing a technique for obtainingneurological and neurophysiological data. At 601, a protocol isgenerated and stimulus is provided to one or more subjects. According tovarious embodiments, stimulus includes streaming video, media clips,printed materials, individual products, etc. The protocol determines theparameters surrounding the presentation of stimulus, such as the numberof times shown, the duration of the exposure, sequence of exposure,segments of the stimulus to be shown, etc. Subjects may be isolatedduring exposure or may be presented materials in a group environmentwith or without supervision. At 603, subject responses are collectedusing a variety of modalities, such as MEG, ERP, EOG, GSR, etc. In someexamples, verbal and written responses can also be collected andcorrelated with neurological and neurophysiological responses. At 605,data is passed through a data cleanser to remove noise and artifactsthat may make data more difficult to interpret. According to variousembodiments, the data cleanser removes MEG electrical activityassociated with blinking and other endogenous/exogenous artifacts.

At 611, intra-modality response synthesis is performed to enhanceeffectiveness measures. According to various embodiments, dipolelocalization measurements are performed to allow improved spatialresolution of brain activity. In particular embodiments, MEG providesenhanced dipole localization and allows determination of temporal andspatial locations of neural activity. At 613, cross-modality responsesynthesis is performed to further enhance effectiveness measures. Itshould be noted that in some particular instances, one type of synthesismay be performed without performing the other type of synthesis. Forexample, cross-modality response synthesis may be performed with orwithout intra-modality synthesis. At 615, a composite enhancedeffectiveness estimate is provided. At 621, feedback is provided to theprotocol generator and presenter device for additional evaluations. Thisfeedback could be provided by the cross-modality response synthesizer orother mechanisms.

According to various embodiments, various mechanisms such as the datacollection mechanisms, the intra-modality synthesis mechanisms,cross-modality synthesis mechanisms, etc. are implemented on multipledevices. However, it is also possible that the various mechanisms beimplemented in hardware, firmware, and/or software in a single system.FIG. 7 provides one example of a system that can be used to implementone or more mechanisms. For example, the system shown in FIG. 7 may beused to implement a data cleanser device or a cross-modality responsessynthesis device.

According to particular example embodiments, a system 700 suitable forimplementing particular embodiments of the present invention includes aprocessor 701, a memory 703, an interface 711, and a bus 715 (e.g., aPCI bus). When acting under the control of appropriate software orfirmware, the processor 701 is responsible for such tasks such aspattern generation. Various specially configured devices can also beused in place of a processor 701 or in addition to processor 701. Thecomplete implementation can also be done in custom hardware. Theinterface 711 is typically configured to send and receive data packetsor data segments over a network. Particular examples of interfaces thedevice supports include host bus adapter (HBA) interfaces, Ethernetinterfaces, frame relay interfaces, cable interfaces, DSL interfaces,token ring interfaces, and the like.

In addition, various very high-speed interfaces may be provided such asfast Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces,HSSI interfaces, POS interfaces, FDDI interfaces and the like.Generally, these interfaces may include ports appropriate forcommunication with the appropriate media. In some cases, they may alsoinclude an independent processor and, in some instances, volatile RAM.The independent processors may control such communications intensivetasks as data synthesis.

According to particular example embodiments, the system 700 uses memory703 to store data, algorithms and program instructions. The programinstructions may control the operation of an operating system and/or oneor more applications, for example. The memory or memories may also beconfigured to store received data and process received data.

Because such information and program instructions may be employed toimplement the systems/methods described herein, the present inventionrelates to tangible, machine readable media that include programinstructions, state information, etc. for performing various operationsdescribed herein. Examples of machine-readable media include, but arenot limited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks and DVDs;magneto-optical media such as optical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory devices (ROM) and random access memory (RAM).Examples of program instructions include both machine code, such asproduced by a compiler, and files containing higher level code that maybe executed by the computer using an interpreter.

Although the foregoing invention has been described in some detail forpurposes of clarity of understanding, it will be apparent that certainchanges and modifications may be practiced within the scope of theappended claims. Therefore, the present embodiments are to be consideredas illustrative and not restrictive and the invention is not to belimited to the details given herein, but may be modified within thescope and equivalents of the appended claims.

1. A system, comprising: a data collection mechanism operable to obtainresponse data from a subject exposed to marketing stimuli, the responsedata obtained using a plurality of modalities includingmagnetoencephalography (MEG); a data cleanser mechanism operable tofilter the response data; an intra-modality response synthesizeroperable to enhance response data associated with each of the pluralityof modalities, wherein enhancing MEG response data comprises performingdipole localization measurements; a cross-modality response synthesizeroperable to combine response data associated with each of the pluralityof modalities to validate response data in each of the plurality ofmodalities; a composite enhanced effectiveness estimator operable togenerate a composite enhanced effectiveness measure of the marketingstimuli.
 2. The system of claim 1, wherein the plurality of modalitiesfurther comprises Galvanic Skin Response (GSR).
 3. The system of claim1, wherein the plurality of modalities further comprisesElectrocardiograms (EKG).
 4. The system of claim 1, wherein theintra-modality response synthesizer enhances response data associatedwith MEG by analyzing data in subbands within the different frequencyranges including delta, theta, alpha, beta, low gamma, and high gamma(kappa) bands.
 5. The system of claim 1, wherein the cross-modalityresponse synthesizer combines response data to validate and enhanceresponse data from one modality using response data from anothermodality.
 6. The system of claim 5, wherein the response data from onemodality is time shifted prior to combination with response data fromanother modality.
 7. The system of claim 1, wherein mechanisms such astime and phase shifting, correlating, and validating intra-modality andcross-modality determinations allow generation of a composite outputcharacterizing the effectiveness of various stimuli.
 8. The system ofclaim 1, wherein brain activity is measured to determine temporal andspatial locations of brain activity.
 9. The system of claim 8, whereinbrain activity is measured to determine interactions and types ofinteractions between various brain regions.
 10. The system of claim 1,wherein different frequency bands and subbands used for multi-regionalcommunication in the brain are measured to determine effectiveness ofstimuli.
 11. The system of claim 1, wherein inter-frequency coupling inthe signals, including signals modulated on a carrier wave, is measuredto determine effectiveness of stimuli.
 12. The system of claim 1,wherein inter-hemispheric time-frequency measurements, includingspecific band and/or sub-band powers, asymmetries in inter-regionalintra-hemispheric coherences, and asymmetries in inter-regionalintra-hemisphere inter-frequency coupling are analyzed to determineeffectiveness of stimuli.
 13. The system of claim 1, wherein combinedtime-frequency responses of ongoing MEG are determined to extract singletrial evoked information
 14. The system of claim 1, wherein individualspecific facial emotion encoding templates are matched with individualresponses to enhance the identification of subject emotional response.15. A method, comprising: providing marketing and/or entertainmentstimuli to a subject; obtaining response data using a plurality ofmodalities, the plurality of modalities including magnetoencephalography(MEG) and electrooculography (EOG); filtering the response data;enhancing response data associated with each of the plurality ofmodalities, wherein enhancing MEG data comprises performing dipolelocalization measurements; combining response data associated with eachof the plurality of modalities to validate response data in each of theplurality of modalities; generating a composite enhanced effectivenessmeasure of the marketing and/or entertainment stimuli.
 16. The method ofclaim 15, wherein the plurality of modalities further comprises GalvanicSkin Response (GSR).
 17. The method of claim 15, wherein the pluralityof modalities further comprises Electrocardiograms (EKG).
 18. The methodof claim 15, wherein the plurality of modalities further comprisespupillary dilation.
 19. The method of claim 15, wherein the plurality ofmodalities further comprises eye tracking,
 20. The method of claim 15,wherein inter-hemispheric time-frequency measurements, includingspecific band and/or sub-band powers, asymmetries in inter-regionalintra-hemispheric coherences, and asymmetries in inter-regionalintra-hemisphere inter-frequency coupling are analyzed to determineeffectiveness of stimuli.
 21. The method of claim 15, wherein combinedtime-frequency responses of ongoing MEG are determined to extract singletrial evoked information
 22. The method of claim 15, wherein individualspecific facial emotion encoding templates are matched with individualresponses to enhance the identification of subject emotional response.23. An apparatus, comprising: means for obtaining response data using aplurality of modalities, the plurality of modalities includingmagnetoencephalography (MEG) and electrooculography (EOG); means forfiltering the response data; means for enhancing response dataassociated with each of the plurality of modalities, wherein enhancingMEG data comprises performing dipole localization measurements; meansfor combining response data associated with each of the plurality ofmodalities to validate response data in each of the plurality ofmodalities; means for generating a composite enhanced effectivenessmeasure of the entertainment stimuli.